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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]With respect to a video-delivery-through-the-lntemet server, especially 
this invention relates to the disposal method in the case of distributing the image of real time 
via a distributing server, when distributing an image file from the middle. 
[0002] 

[Description of the Prior ArtjAccording to the request from a terminal, an Image or an audio (an 
"audio" Is hereafter used in the sense of a sound or an audio), and an image via a transmission 
line. In distributing corresponding data to a terminal from a server, the system layer which 
multiplexes synchronization information for the reproduction timing of each media. I.e., an 
Image, and an audio to be shown, and picture-image-data audio Information and 
synchronization Information as one data is needed. It Is an eye conventionally as a system 
which specifies these system layer and synchronization infomnation. S Ore/eye E There was a 
file fomriat (following MP4 fomiat) defined by See ISO/IEC 14496-1. MP4 fomiat comprises the 
attendant information portion 1 1 called moov like drawing 1 , and media data (picture image 
data or audio information portion 12) which is called mdat and which were coded, moovl 1 
comprises header information and each media information (following header information 13), a 
storing position of each media, and a regeneration time infomiation (time stamp) portion 
(henceforth, media access Information 14), as further shown In drawing 2 . The number of the 
Images Included In subsequent data, image size, a coding mode, the bit rate, etc. are 
described by the header Information 13, for example, on the other hand - the media access 
information 14 - the reproduction unit (henceforth access unit : AU) of the image (or audio) 
data in mdat12 - each time - storing position information and the regeneration time 
information on each AU are stored. 

[0003]ln the case of a file of MP4 format like drawing 1 , distribute a file via a transmission line 
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and are parallel with receiving operation from the file reception middle at the received terminal, 
When the operation which reproduces an image Is considered, it is necessary to read all the 
data of moov1 1 portion which is not used for head part reproduction of a file, and the time 
delay from a file receiving start to a reproduction start Increases. In order to reduce the time 
delays In such a case, contents are subdivided to short-time contents like drawing 3 , The 
method of distributing and arranging the media access Information and media data 
corresponding to each short-time contents In a file by turns, I.e., moov21 of a head, two or 
more moof23, and the method of distributing and arranging to 25 are known. The structure of 
moov when distributing moov to one moov and one or more moof(s) has become like drawing 
4, and when the information 32 about moot exists, it shows the time moot exists henceforth. 
The structure of moot comprises media access information (a data position and a time stamp) 
over each media contained In mdat which follows the consecutive numbers 41 of the relevance 
moof, and the relevance moof like drawing 5 . 
[0004] 

[Problem to be solved by the lnventlon]ln the case of the stream encoded In real time, the 
above-mentioned conventional technology could not be directly adapted when reproducing 
from the middle of contents. An object of this Invention Is to provide the stream converting 
method which makes refreshable the stream which Is reproduced from the middle of contents 
or is encoded by real time, and which continues Indefinitely In the temilnal only con-esponding 
to the system of conventional technology. 
[0005] 

[Means for solving problem]To achieve the above objects, the moof portion of an original 

stream is changed into moov in a distributing server. 

[0006] 

[Mode for carrying out the InventlonjThe 1st wori<lng example by this Invention is shown in 
drawing 6 below. Drawing 6 generates the original stream 53 which once agreed the Image 51 
In the MP4 fomiat by the encoding processing 52, Once accumulating by the accumulation 
processing 54, receive the reproduction start time 60, read the accumulated stream 55, and by 
the conversion process 56. The converted stream 57 started from the time of the neighborhood 
where it was specified In the middle of the stream 53 Is generated, and the generated stream 
57 is distributed to a terminal as the distribution stream 59 by the message distribution 
processing 58. 

[0007] Drawing 7 shows the relation of the original stream 53 and the converted stream 57 in 
the above-mentioned conversion process. The original stream 53 is arranged at the head by 
mdat22 which follows moov21 and it, and Following moof23/mdat24 and moof25/mdat26. The 
combination of moof and mdat corresponding to It Is repeated (In the following explanation, the 
group of the media data mdat and moof corresponding to them Is expressed using T like 
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moof/mdat). In [ when AU applicable to start time is contained in mdat24, make the new 
starting point into the head of mdat24, and ] the conversion process 56, New moovTO is 
generated from the information on each media (an image and an audio) indicated to moov21 , 
and the information on mdat24 indicated to moof23. Henceforth, mdat of the original stream 53 
is copied mdat24 and 26, and moof25 Is outputted as moof71 , after consecutive numbers are 
changed. A stream Is a stream which skipped the portion which Is unnecessary to generation 
of the stream 57 among the streams 53 as for 55, for example, mdat22 grade. 
[0008] Drawing 8 is the flow chart which showed the details of the conversion process 
mentioned above. In the conversion process 56, first, moov21 of an original stream is read and 
the header information described there is read. Next, moof23 used as a new stream starting 
position is searched. In the moov output process 80, new moov70 is outputted using the 
above-mentioned header information and the information on moot used as a stream starting 
position. It goes into the loop processing 81 which outputs the combination of moof/mdat of a 
predetermined number, after continuing moovZO and outputting mdat24 corresponding to 
moof23. In loop processing, the end decision 82 of whether there Is any combination of 
moof/mdat which continues first, and a stream Is performed. In the end of a stream. It shifts to 
the processing 83, and Infomiatlon, Including the data size of the contents currently written In 
moovZO, reproducing time, etc.. Is updated, and It ends processing. On the other hand. In not 
ending, processing by the side of the loop 81 Is perfomied by the end decision processing 70. 
That is, the main force of the moof corrected after reading the next moof and correcting 
consecutive numbers to a new value is carried out, and corresponding mdat is outputted after 
that. After performing these processings, end decision processing 70 Is performed again. 
When the end of conversion outputs the final data of (1) contents, a terminating request comes 
from (2) terminals, and a server is independently completed by timeout of (3) distribution error 
and the response from a terminal, etc., ** exists. 

[0009 ]Drawlng 9 Is a flow chart for explaining the details of the moov output process 80 of 
drawing 8 . In the Moov prime processing, the header Infomiatlon of already read moov21 Is 
outputted first. Next, the synchronous correction Information for amending the regeneration 
time of each media Is outputted. Then, the media access Information on head moof23 Is 
outputted. The number of bytes Is calculated and all of these output data are Indicated at the 
moov head as size of moov70. 

[0010] Drawlng 10 Is a figure for explaining the contents of the output process 85 of the 
synchronous con-ectlon infomiatlon of drawing 9 . Drawing 10 shows synchronous amendment 
when what comprised two media, an audio and video, as an original stream is changed into 
the stream started from the middle. Generally, in the audio and video in one contents, since 
the sampling time of each AU is asynchronous, when a stream Is extracted from the original 
stream middle, the regeneration time of top audio AU and the regeneration time of head video 
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AU are not in agreement. That is, lil<e drawing 10 , when malting the point near the boundary of 
video AU2 and video AU3 into the new starting point, as shown in a figure, AU3 to an audio will 
be started from AU8 and the time lag T produces video. In a synchronization correction 
process, it becomes possible by describing the value of this T to a converted stream to make 
the time relation of an audio and video into the same time position as an original stream in a 
terminal at the time of playback. In drawing 10 , since video is behind, the information which 
shows "T Delay the playback start of a video signal" is outputted, but the information on the 
purport "the playback start of an audio signal is delayed predetermined time" when the audio 
can be sent is outputted. Since all of such regeneration time are described by moov or moof of 
the original stream, it perfonns judgment and calculation by the size of the value of the time 
stamp described by moov or moof of the original stream. 

[001 1]To one constant [ the regeneration time of AU of an audio ] for example, in a cycle of 30 
ms, and short, the regeneration time of AU of video is as long as ten frames per second, i.e., 
100 ms, it is a variable frame rate further, and, in the case of coding of a low rate, the cycle 
becomes irregular in many cases. For this reason, in the terminal side, it reproduces by 
making an audio cycle into a reference cycle In many cases. That is, in the terminal side, only 
when regeneration of video is needed on the basis of regeneration of an audio signal, the 
composition which perfomis video output processing is taken. Therefore, if playback of the 
audio used as a standard is started, it can be simultaneous or the burden by the side of a 
terminal can be eased by [ display video later, namely, "delays the playback start of a video 
signal" ] taking a reproduction starting position like. On the contrary, if audio reproduction is 
delayed, regeneration of a video signal will have to be made to start before the start of 
processing used as a standard, it will be the control method which differs in the processing at 
the time of a start from the usual processing, and additional software or hardware will be 
needed for the terminal side. As mentioned above, the stream started from the middle of the 
original contents by wori<ing example described by drawing 10 f rom drawing 6 , The server side 
can realize generation of new moov in few throughputs of only fine correction of moof, and, on 
the other hand, the terminal side can realize reproduction from the middle the same at all with 
the processing which carries out reception reproduction of the conventional storm. 
[0012]lf especially the composition of drawing 8 is combined with a video server, when 
contents will be distributed from a contents head, It can realize by [ as the case where it 
distributes from the contents middle / same ] moreover holding only single data, and is 
effective in the ability to make capacity of the storage device of a video server small, or hold 
more data with a fixed storage device. 

[0013] Drawing 11 shows the 2nd working example of this invention. Drawing 1 1 is processing 
constitution which starts the original stream (image of real time) coded by real time from 
arbitrary points in time, and is distributed as a new converted stream. The inputted image 51 is 
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encoded by the encoding processing 52 in real time, and the original stream 53 is generated. 
In the real-time-data conversion process 101, this original stream 53 is changed into the 
converted stream 57 started from the time of there being directions in real time, and the 
message distribution processing 58 distributes. This is used for a use which accesses the 
image currently photoed at any time from two or more terminals with a surveillance camera etc. 
like drawing 12 , for example. 

[0014] Drawing 13 Is a flow chart explaining the details of the real-time-data conversion process 
of drawing 1 1 . First, before a conversion start, the header Information of the stream 53 Is 
acquired and accumulation processing 120 Is perfomied. Next, when there are not waiting and 
a distribution request about the distribution request from a temiinal, next moof or mdat of the 
stream 53 is searched. 

[0015]When a distribution request occurs, the first moof after a demand is searched, and moov 
is generated and outputted from this moof information and previous header information. 
Subsequent processings are the same as drawing 8 . The case where end processing of a real- 
time operation "Is not ended" In addition to end processing in the case of drawing 8 Is Included. 
Therefore, It Is necessary to Indicate the information showing the meanings, such as "unfixed", 
"Infinity", and "real time distribution", in the fields indicated at the head moov, such as 
reproducing time (contents length) of data size and contents, and to carry out the case where 
data size Is limited and distinction like [ in the case of drawing 8 ] in them. It is possible to omit 
the correction 83 of moov description data by indicating the information showing the meanings, 
such as "unfixed", "infinity", and "real time distribution", in the fields indicated at the head 
moov, such as data size and reproducing time, also in the case of drawing 8 . 
[0016]By storing temporarily mdat which moof(s) and corresponds in a buffer etc., The time 
delay of an original stream and a distribution stream becomes large, although real time nature 
Is spoiled a little, from the distribution request issue from a terminal, the time lag of a video- 
delivery-through-the-lntemet start can be compensated, and the image at the distribution 
request time to distribution can be carried out. 

[0017]The following processings may be sufficient, although header information is acquired 
and acquisition of header information is set up in the accumulation processing 120 at the time 
of an encoding start. 

(1) Distribute header information periodically with the channel put side by side to the inside of 
the stream 53, or the stream 53. 

(2) From a real-time conversion process part, ask an encoder part header infomnation and the 
channel put side by side to the inside of the stream 53 or the stream 53 for every inquiry 
notifies header information to it by an encoder part. 

(3) Beforehand, record header information on the real-time conversion process part, and 
process with the same parameter as this by an encoder part. 
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[0018] Drawing 14 is a moov output process for real time data. In order to secure real time 
nature, the output of moov to generate is made into a temporary buffer, and although the 
contents of processing are the same as drawing 9 , immediately after moov generation is 
completed, moov data is distributed by the data prime processing 125 in a buffer. 
[0019] Drawing 15 is the 3rd working example of this invention. In drawing 15 , the stream of 
plurality (the example of drawing 16 t hree) from which the bit rate differs is prepared about the 
same contents, and transmission with the variable bit rate is attained by changing two or more 
streams In the combination unit of moof/mdat according to the demand from a terminal. 
Drawing 16 is the example applied to the system that the bit rate of the circuit to a tenninal is 
changed, as an example with the variable bit rate, and it becomes possible to distribute in the 
bit rate which was adapted for the network bit rate. 

[0020]ln drawing 16 , with the distribution started at 32k bps at the beginning, expansion of the 
bandwidth from the time 4 neighborhood has a change request to 48k bps in the midst of the 
time 4, and the bit rate is changed [ time /II] into 32k bps for the rate from the time 1 3 from 
the time 5 change and henceforth at 64kpbs. 

[0021] Drawing 17 is a flow chart for explaining the details of the real-time-data conversion 
process corresponding to processing of drawing 15 . Although the contents of processing are 
almost the same as drawing 13 , it differs from drawing 13 in that the bit rate change 
processing 152 is added before the bit rate setting processing 150 and each moof/mdat 
distribution at the time of a distribution start In the bit rate change-request existence judging 
151 and with a bit rate change request. Each moof/mdat is read from the stream 
corresponding to the bit rate set up at each time. On the other hand, about the stream of the bit 
rate which does not correspond above, the synchronization is taken for moof/mdat in the skip 
and the usual state by the processing 160,161. Although the 3rd working example of this 
Invention explained based on the 2nd working example. It Is clear for combination to be 
possible for the 1st working example of this Invention. 

[0022]lt becomes possible from drawing 15 t o distribute with the rate which suited the 
occasional line rate by processing of drawing 17 also In the system by which the line rate to a 
terminal Is changed. Although It Is small, change of the line rate under distribution Is effective, 
also when a actual line rate Is determined by ambient environment etc. and Is not determined a 
priori. Measurement Division of a line rate Is performed using the Information on the following 
environment. 

[0023](1) Notify from a tenninal the receiving bit rate measured at the tenninal. 
(2) Notifying the information relevant to a line rate from a terminal, the distribution side sets up 
the bit rate suitable based on the received information. For example, in an acquisition number 
of circuit and a radio channel, the intensity of an electric wave, the value of an error rate, etc. 
are used by the channel which bundles multiple-lines, such as a maximum bit rate. 
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(3) A server and a terminal synchronize, operate, namely, when it is a system by which the 
notice of data receiving completion or the Request to Send of next data is obtained from the 
terminal side, measure the transmitting bit rate in a server. 

(4) When a server and a temiinal synchronize and it operates, presume the transmitting bit 
rate from the residue of a transmission buffer. 

(5) A communication bit rate notice is given from a networl<. 

(6) The information relevant to a line rate is notified from a network. 

(7) The above should put together. 

[0024] Drawing 18 is a figure explaining the outline of the 4th working example of this invention. 
The 4th working example is a modification of the 1st working example of drawing 6 , and in the 
1 st working example, for the stream which used moof of the form of drawing 8 , although, it is 
aimed at the stream which does not use moof of drawing 1 by the 4th working example. 
[0025]ln the 4th working example, since the original stream to input is the form of drawing 1 , 
the contents become like the upper part of drawing 18 . That is, there is only one moov and the 
inside of it consists of the header information 1 3 and the media access information 14. hHere, 
the media access information 14 is logically divided into the data position and time stamp for 
every fragmentation data which comprises short time, and can be considered. When 
reproduction start time is specified, moovTO of a converted stream is generated from the media 
access information 201 con-esponding to the time, and the header infomiation 13. The 
applicable fragmentation data 202 is outputted as mdat24. The fragmentation data in mdat12 
and the media access information in moovl 1 applicable to it are outputted one by one 
henceforth. 

[0026] Drawing 19 is a flow chart for explaining processing of the 4th working example. 
Although fundamental processing is the same as processing of drawing 8 , in drawing 19 , 
media access infomiation 201 retrieval processing 210 of start data is perfomied instead of 
moof23 search of drawing 8 . In drawing 8 , mdat obtained by perfomiing start data 202 search 
21 1 is outputted as head mdat24 by drawing 19 t o outputting head mdat24 as it is based on a 
data position obtained in the processing 210. 

[0027]l-lereafter, in the loop 220, the media access information read-out 212 and next data 
read-out processing 214 in mdat are performed similarly. In drawing 8 , although only 
consecutive numbers were corrected and read moof was outputted, when it is drawing 19 , in 
the processing 213, moof is generated and outputted from the corresponding data of moov. 
[0028]Thus, application is possible also to the stream for which this invention is not using 
moof. If it applies to the stream which is not using moof, the analysis of the data in moov will 
be needed and a throughput will be cut in many compared with the case where moof is used. 
On the other hand, when moof was being used, the starting point was set up only in the unit of 
moof, but in the 4th working example, it can start from arbitrary AU. IHowever, start AU needs 



http://www4.ipdl.inpit.go.ip/cgi-bin/tran web cgi eiie?atw u=http%3A%2F%2Fwww4.ip... 5/26/2011 



JP,2003-114845,A [DETAILED DESCRIPTION] 



Page 8 of 8 



to be AU in which random access is possible. The stream which mal<es the starting point AU In 
the middle of moot Is generable to the stream which uses moot by applying the 4th working 
example and performing processing which analyzes the Inside of moov or moof. 
[0029]When It combines with a video server from the 2nd working example like the 1st working 
example to the 4th working example, there are the following effects. In the 2nd working 
example, real time data can be distributed from arbitrary moof/mdat. Since there are few 
throughputs of conversion. It becomes possible to distribute simultaneously In limited CPU to 
more terminals In which distribution starting positions differ, respectively. 
[0030]ln the 3rd wori^lng example, in order to change the bit rate at the time of the distribution 
from a video server. Install a transformer codec (inverter which combined the decoder and the 
encoder), or, Incompressible contents are prepared and distribution con-esponding to zone 
change can be realized in a very small throughput compared with distributing carrying out 
encoding processing to real time for every terminal. Since processing of them is possible in the 
same throughput even If change of the zone for every terminal differs, respectively, the 
terminal number which can be processed In CPU of a fixed throughput Is not changed. 
[0031] 

[Effect of the lnventlon]ln the 1 st wori<lng example, the stream started from the middle of the 
original contents. The server side Is realizable In few throughputs of only generation of moov, 
and fine correction of moof, and on the other hand, the temrilnal side can realize reproduction 
from the middle, without changing an old storm In any way with the processing which carries 
out reception reproduction. In the 2nd working example, the stream coded by real time is 
started from arbitrary points In time. Processing distributed as a new stream can be realized in 
few throughputs, and, on the other hand, the terminal side can be reproduced In the Image of 
real time the same at all with the processing which carries out reception reproduction of the old 
storm. 

[0032]ln the 3rd wori<lng example. It becomes possible to distribute with the rate which suited 
the occasional line rate also In the system by which the line rate to a temnlnal Is changed. 
Although It Is small, change of the line rate under distribution Is effective, also when a actual 
line rate Is determined by ambient environment etc. and Is not determined a priori. 



[Translation done.] 
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